Information processing device, information processing system, information processing method, and program

ABSTRACT

An information processing device, an information processing system, an information processing method, and a program capable of appropriately reducing the amount of information of an image are provided. An information processing device includes an image acquisitor, an information amount reduction degree determiner, and an information amount reducer. The image acquisitor acquires an image acquired by imaging an actual space. The information amount reduction degree determiner determines a degree of reduction of an amount of information on the basis of an attribute of a subject shown in the image. The information amount reducer generates information reduced data acquired by reducing at least a part of the amount of information of the image in accordance with the degree of reduction of the amount of information.

BACKGROUND OF THE INVENTION Technical Field

Embodiments of the present invention relate to an information processingdevice, an information processing system, an information processingmethod, and a program.

Related Art

Japanese Unexamined Patent Application, First Publication No. 2016-71639disclose a technology for collating individual faces while reducing theinformation used for identifying individuals by calculating andcollating feature quantities of subjects (faces and the like) afterperforming blurring processing for subjects (faces and the like) shownin an image acquired by a monitoring camera.

However, in the technology described above, although privacy isprotected by performing blurring processing for an image, featurequantities are calculated and collated from images of which informationis reduced, and accordingly, there are cases in which collation accuracyof images becomes low.

SUMMARY

An information processing device according to an embodiment includes animage acquisitor, an information amount reduction degree determiner, andan information amount reducer. The image acquisitor acquires an imageacquired by imaging an actual space. The information amount reductiondegree determiner determines a degree of reduction of an amount ofinformation on the basis of an attribute of a subject shown in theimage. The information amount reducer generates information reduced dataacquired by reducing at least a part of the amount of information of theimage in accordance with the degree of reduction of the amount ofinformation.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a system diagram illustrating an overview of an informationprocessing system according to a first embodiment;

FIG. 2 is a diagram illustrating an example in which the amounts ofinformation are uniformly reduced in person images;

FIG. 3 is a diagram illustrating an example in which the amount ofinformation is reduced in a person image according to the firstembodiment;

FIG. 4 is a block diagram illustrating one example of the hardwareconfiguration of an information processing system according to the firstembodiment;

FIG. 5 is a block diagram illustrating one example of the functionalcomponents of a terminal controller and a server controller according tothe first embodiment;

FIG. 6 is a diagram illustrating a setting of association between anappearance frequency of an attribute of a person and a blurringintensity;

FIG. 7 is a flowchart illustrating one example of Re-id processingaccording to the first embodiment;

FIG. 8 is a flowchart illustrating details of blurring processingaccording to the first embodiment;

FIG. 9 is a diagram illustrating an example of a setting of a blurringintensity for each part of a person according to a second embodiment;

FIG. 10 is a flowchart illustrating blurring processing according to thesecond embodiment;

FIG. 11 is a diagram illustrating an example in which a blurringintensity is adaptively changed according to a third embodiment;

FIG. 12 is a flowchart illustrating blurring processing according to thethird embodiment;

FIG. 13 is a diagram illustrating a blurring intensity according to afourth embodiment;

FIG. 14 is a flowchart illustrating blurring processing according to thefourth embodiment;

FIG. 15 is a diagram illustrating a blurring intensity according to afifth embodiment;

FIG. 16 is a block diagram illustrating one example of the functionalconfigurations of a terminal controller and a server controlleraccording to the fifth embodiment;

FIG. 17 is a flowchart illustrating blurring processing according to thefifth embodiment;

FIG. 18 is a diagram illustrating an appearance frequency of a personaccording to a sixth embodiment;

FIG. 19 is a system diagram illustrating an overview of an informationprocessing system according to the sixth embodiment;

FIG. 20 is a block diagram illustrating one example of the functionalconfigurations of a terminal controller and a server controlleraccording to the sixth embodiment; and

FIG. 21 is a flowchart illustrating blurring processing according to thesixth embodiment.

DETAILED DESCRIPTION OF THE INVENTION

Hereinafter, an information processing device, an information processingsystem, an information processing method, and a program according toembodiments will be described with reference to the drawings.

First Embodiment

First, an overview of an information processing system according to thisembodiment will be described.

FIG. 1 is a system diagram illustrating an overview of the informationprocessing system 1 according to this embodiment. The informationprocessing system 1 is a system that associates the same personscaptured by cameras (Person Re-identification (Re-id)) by calculatingand collating feature quantities of persons from images captured by aplurality of the cameras 10 (10-1, 10-2, 10-3, . . . ) installed at astation yard, the inside of a commercial facility, a shopping district,or the like. Each of terminals 20 (20-1, 20-2, 20-3, . . . ) connectedto the cameras 10 calculates feature quantities of subjects (persons andthe like) shown in an image captured by the camera 10 and transmits thecalculated feature quantities to a server 30. The server 30 receivesfeature quantities transmitted from each terminal 20. The server 30collates feature quantities received from the terminals 20 andassociates feature quantities determined as in the same person with eachother on the basis of collation results (a degree of similarity). Inthis way, the information processing system 1 can detect behaviors ofthe same person and can collect statistical information such as featuresand a behavior pattern of each person in a yard, inside a commercialfacility, on a shopping street, and the like.

For example, as illustrated in the drawing, it is assumed that a personU moves in an order of points A, B, and C. The person U at the time ofbeing present at the spot A is shown in an image captured by the camera10-1. The person U at the time of being present at the spot B is shownin an image captured by the camera 10-2. The person U at the time ofbeing present at the spot C is shown in an image captured by the camera10-3. The terminal 20-1 calculates a feature quantity of the person Ufrom the image in which the person U at the time of being present at thespot A is shown captured by the camera 10-1 and transmits the calculatedfeature quantity to the server 30 in association with capture timeinformation. The terminal 20-2 calculates a feature quantity of theperson U from the image in which the person U at the time of beingpresent at the spot B is shown captured by the camera 10-2 and transmitsthe calculated feature quantity to the server 30 in association withcapture time information. The terminal 20-3 calculates a featurequantity of the person U from the image in which the person U at thetime of being present at the spot C is shown captured by the camera 10-3and transmits the calculated feature quantity to the server 30 inassociation with capture time information. The server 30 collates thefeature quantities transmitted from the terminals 20 and associatesfeature quantities determined to be the same person with each other. Inthis way, the information processing system 1 can detect a behavior of acertain same person (here, the person U). In addition, the informationprocessing system 1 does not perform identification of individuals foridentifying a person who is represented by the feature quantitiesassociated as the same person.

Meanwhile, as described above, in the technology of determining whetherpersons are the same or not by collating feature quantities of subjects(persons and the like) captured by the cameras 10, it is necessary tocollect feature quantities of subjects (persons and the like) and atleast temporarily store the collected feature quantities in a storagemedium such as a hard disk or a memory, and a high security level needsto be set therefor from the point of view of privacy protection.However, a high cost is incurred for building a high-securityenvironment. While a high-security environment needs to be built in, inthe case of opening/closing an airport gate using face authentication,account management using fingerprint authentication, and the like, in acase in which an individual does not need to be identified, and, forexample, collection of statistical information of features, behaviorpatterns, and the like of visitors for a facility is a target, there isa problem that the cost of building a high-security environment thereforis high.

Thus, the terminal 20 calculates a feature quantity from informationreduced data that is acquired by reducing the amount of information suchthat an individual cannot be identified. For example, the terminal 20performs an information amount reducing process with a predetermineddegree of reduction of the amount of information for images of subjects(persons and the like) captured by the cameras 10, calculates featurequantities of the subjects (persons and the like) from the informationreduced data after reduction of the amount of information, and transmitsthe calculated feature quantities to the server 30. In this way, anindividual cannot be identified using information (feature quantities)that is transmitted from the terminals 20 to the server 30 and is storedin the server 30, and accordingly, a system that takes privacy intoconsideration even when a security level of a storage medium storingfeature quantities is low can be configured.

For example, the degree of reduction of the amount of information may bedefined according to a value of a sharpness of an image, a resolution ofan image, a bit depth of an image, a JPEG compression rate of an image,an intensity of a mosaic, image filling, a contrast of an image, abrightness of an image, an aspect ratio of an image, or the like. Theinformation reduced data is acquired by performing an information amountreducing process on an original image by applying a value of the degreeof reduction of the amount of information. Regarding this amount ofinformation of an image, for example, for the sharpness of an image, theresolution of an image, and the bit depth of an image, the amount ofinformation is larger when the value thereof is higher, and the amountof information is smaller when the value thereof is lower. For a JPEGcompression image of an image and the intensity of the mosaic, theamount of information is larger when the value thereof is lower, and theamount of information is smaller when the value thereof is higher. Forthe filling of an image, the amount of information is larger when afilled area is smaller, and the information is smaller when the filledarea is larger. For the contrast of an image and the brightness of animage, the amount of information is larger with appropriate valuesthereof, and the amount of information is smaller with inappropriatevalues thereof. Setting values for which an image has littleoverexposure, is not too dark, and in which objects have a clear outlineis appropriate.

Here, an example in which the resolution of an image is reduced byapplying blurring processing using a moving average filter, a Gaussianfilter, or the like as an information amount reducing process will bedescribed. When a degree of reduction of the amount of information inthe blurring processing is a blurring intensity, it is assumed that thedegree of reduction of the amount of information is higher in a case inwhich the blurring intensity is “strong” than in a case in which theblurring intensity is “weak.”

FIG. 2 is a diagram illustrating an example in which the amounts ofinformation are uniformly reduced in person images. This drawingillustrates an example of a case in which the amount of information isreduced by performing blurring processing uniformly with the sameintensity “strong” for person images of five persons U1, U2, U3, U4, andU5 shown in an image captured by the camera 10. One example of personimages before blurring processing is schematically illustrated in anupper part, and one example of person images after blurring processingis schematically illustrated on a lower part. It is assumed thatclothing worn by the person U1 is green, clothing worn by the person U2is red, and clothing worn by the persons U3, U4, and U5 is blue. In theimage of the person U1 and the person U2, by performing blurringprocessing, it can be made difficult to identify individuals, and thereis a feature of having different colors of clothing among five personsand the like, and accordingly, even when feature quantities arecalculated from an image after the blurring processing, there is noproblem in the accuracy of collation. Meanwhile, like the persons U3,U4, and U5, in a case in which the number of persons wearing clothing ofthe same color increases, the features of these three persons aresimilar. Thus, in a case in which the blurring intensity is strong, notonly it is difficult to identify individuals, but also collationaccuracy decreases in an image after blurring processing. In this way,in a case in which the blurring intensity is uniformly strong, althoughthe privacy is taken into account, there is a problem in that thecollation accuracy decreases. For this reason, it is necessary toappropriately set the blurring intensity in accordance with a person. Anappropriate setting value of the blurring intensity changes inaccordance with a hairstyle, a color, clothing, a place, and the like ofa target person. For example, in a case in which a person is identifiedusing clothing included in a person image, in a scene such as an officein which many persons are wearing suits, it is difficult to identify aperson using clothing. For example, it is necessary to inhibit adecrease in the collation accuracy by setting the blurring intensity tobe weak. On the other hand, in a scene in which some persons are wearingconspicuous clothing in a commercial facility or the like, there is alikelihood of being able to easily identify a person from clothing. Insuch a case, it is necessary to focus on taking privacy intoconsideration by setting the blurring intensity to be strong.

Thus, in this embodiment, the blurring intensity is changed inaccordance with whether it is easy or difficult to identify a personwithin an image. For a person image of a person who can be easilyidentified, by setting the blurring intensity to be strong, the privacyis taken into consideration. On the other hand, for a person image of aperson who cannot be easily identified, by setting the blurringintensity to be weak, a decrease in the collation accuracy is inhibited.By appropriately setting the blurring intensity for each person image, adecrease in the collation accuracy for persons of which features aresimilar is inhibited, and decrease in the collation accuracy can bereduced such that it is minimal while the privacy is taken intoconsideration.

FIG. 3 is a diagram illustrating an example in which the amount ofinformation is reduced in a person image according to this embodiment.In this diagram, the blurring intensity of blurring processing for animage of persons U3, U4, and U5 whose colors of clothing are the same isset to be weaker than the blurring intensity of blurring processing foran image of persons U1 and U2 whose colors of clothing are differenteach other, which is different from the example illustrated in FIG. 2 .For example, in a case in which there are no persons (or there is asmall number of persons) whose colors of clothing are the same,identification of an individual can be easily performed, andaccordingly, the information processing system 1 sets the blurringintensity to be strong. On the other hand, in a case in which there aremany persons whose colors of clothing are the same, it is difficult toperform identification of an individual, and accordingly, theinformation processing system 1 sets the blurring intensity to be weak.

In this way, the information processing system 1 sets the degree ofreduction of information to be strong for an image of persons (in otherwords, persons for which individuals can be easily identified) in whichan appearance frequency of persons similar to each other is low, andaccordingly, it becomes difficult to identify individuals, and highcollation accuracy can be secured. On the other hand, the informationprocessing system 1 sets the degree of reduction of information to beweak for an image of persons (in other words, persons for whichindividuals inherently cannot be easily identified) in which anappearance frequency of persons similar to each other is high, andaccordingly, a decrease in the collation accuracy can be inhibited. Inother words, the information processing system 1 can appropriatelyreduce the amount of information of an image, and both protection ofprivacy and high accuracy of collation of persons can be achieved.

Hereinafter, the configuration of the information processing system 1will be described in detail.

FIG. 4 is a block diagram illustrating one example of the hardwareconfiguration of an information processing system 1 according to thisembodiment. In this drawing, the same reference signs are assigned tocomponents corresponding to FIG. 1 . The information processing system 1includes a plurality of cameras 10 (10-1, 10-2, . . . , 10-n; here n isa natural number), terminals 20 (20-1, 20-2, . . . , 20-n; here, n is anatural number) that are respectively connected to the cameras 10 in awired or wireless manner, and a server 30. The camera 10, for example,is a video camera, a monitoring camera, a network camera, or the like,captures an image in an actual space at a predetermined field angle, andtransmits the captured image to the terminal 20. The image may be eithera still image or a moving image. The terminal 20 and the server 30 areinformation processing devices including computers. The terminal 20 andthe server 30 are connected through a communication network. Inaddition, the camera 10 and the terminal 20 may be integrallyconfigured.

The terminal 20 includes a communicator 21, an input 22, a display 23, astorage 24, and a terminal controller 25. The server 30 includes acommunicator 31, an input 32, a display 33, a storage 34, and a servercontroller 35. Each of the communicator 21 and the communicator 31 isconfigured to include a digital input/output port such as an Ethernet(registered trademark) port or a universal serial bus (USB) or radiocommunication such as Wi-Fi (registered trademark), and the like. Thecommunicator 21 and the communicator 31 perform communication through acommunication network on the basis of control according to the terminalcontroller 25 and the server controller 35.

Each of the input 22 and the input 32 is configured to include akeyboard, a mouse, a touch pad, or the like. The input 22 and the input32 respectively output operation signals representing input operationsto the terminal controller 25 and the server controller 35 on the basisof users' operations on a keyboard, a mouse, a touch pad, or the like.In addition, each of the input 22 and the input 32 may be configuredintegrally with a display as a touch panel.

Each of the display 23 and the display 33 is a display that displaysinformation such as an image or text and, for example, is configured toinclude a liquid crystal display panel, an organic electroluminescence(EL) display panel, or the like. In addition, the display 23 and thedisplay 33 may be configured respectively as bodies separate from theterminal 20 and the server 30 and, for example, may be external-typedisplay devices.

The storage 24 and the storage 34, for example, include a hard diskdrive (HDD), a solid state drive (SSD), an electrically erasableprogrammable read-only memory (EEPROM), a read-only memory (ROM), arandom access memory (RAM), and the like and store various types ofinformation, images, programs, and the like respectively processed bythe terminal controller 25 and the server controller 35. In addition,the storage 24 and the storage 34 are not limited to be respectivelybuilt into the terminal 20 and the server 30 and may be external-typestorage devices connected using digital input/output ports such as USBsor the like.

The terminal controller 25 is configured to include a central processingunit (CPU) and controls each unit of the terminal 20 by executingvarious kinds of programs stored in the storage 24. For example, theterminal controller 25 acquires an image captured by the camera 10through the communicator 21 and performs a process (for example,blurring processing) of reducing the amount of information for a personimage shown in the acquired image. In addition, the terminal controller25 calculates a feature quantity from the person image after processingand transmits the calculated feature quantity to the server 30 throughthe communicator 21. The server controller 35 is configured to include aCPU and controls each unit of the server 30 by executing various kindsof programs stored in the storage 34. For example, the server controller35 acquires feature quantities of persons transmitted from a pluralityof terminals 20 through the communicator 31, collates the acquiredfeature quantities, and performs association of the same person. In thefollowing description, when the camera 10, the terminal 20, and theserver 30 perform communication, it will be omitted in the descriptionthat the communication is performed through the communicator 21 and thecommunicator 31.

Next, functional components realized by the terminal controller 25 andthe server controller 35 executing programs will be described in detail.

FIG. 5 is a block diagram illustrating one example of functionalcomponents of the terminal controller 25 and the server controller 35according to this embodiment.

The terminal controller 25 includes an image acquisitor 251, a persondetector 253, a person image acquisitor 254, an attribute detector 257,an information amount reduction degree determiner 258, an informationamount reducer 259, and a feature quantity calculator 260 (a firstfeature quantity calculator). The server controller 35 includes afeature quantity storage 351, a feature quantity collator 352 (a firstcollator), and a display controller 353.

The image acquisitor 251 acquires an image captured by the camera 10.For example, the image acquisitor 251 acquires image data of an imagecaptured by the camera 10. In the following process, an image acquiredby the image acquisitor 251 may be processed in real time online or maybe processed offline after the image is stored in the storage 24.

The person detector 253 detects a person image of a person shown in animage from the image acquired by the image acquisitor 251. For example,the person detector 253 detects a whole body of a person shown in animage, an upper body half of the person, a face of the person, and apart or the whole of clothing and the like of the person. The detectionof a person can be performed using results acquired by a persondetector, instance segmentation, semantic segmentation, and the like.The person detector (a face detector, an upper body half detector, or aperson detector), the instance segmentation, and the semanticsegmentation can be realized using template matching, statistics ofluminance gradient information of an image, deep learning, and the like.In addition, the person detector 253 may detect a person image on thebasis of differences from a background-only image (an image of the samefield angle in which moving bodies such as persons are not present) andthe like. The method of detecting a person image is not limited to thosedescribed above, and an arbitrary detection method can be applied.

The person image acquisitor 254 acquires a person image detected by theperson detector 253. For example, the person image acquisitor 254acquires a person image acquired by cutting out an area of a personimage detected by the person detector 253 from an image acquired by theimage acquisitor 251 and outputs the acquired person image to theattribute detector 257 and the information amount reducer 259.

The attribute detector 257 detects attributes of a person on the basisof a person image acquired by the person image acquisitor 254. Theattributes of a person represents properties and features of a personand are, for example, a color of clothing, a type of clothing (a suit,trousers, half pants, or a skirt, presence/absence of a hat,presence/absence of glasses, and the like), a hairstyle of a person(long hair, short hair, or the like), a color of the hair of a person,the gender of a person, the age of a person, text (a name and the like),and the like. One attribute or a plurality of items may be configured tobe detected. The gender of a person, the age of a person, a hairstyle ofa person, a type of clothing, and the like may be estimated using faceattribute estimation and person attribute estimation (human attributerecognition). The color of clothing and the color of hair may beestimated using color information (chromaticity and the like) of animage. Text (a name and the like) may be estimated using textrecognition. The face attribute estimation, the person attributeestimation, and the text recognition can be realized using templatematching, statistics of luminance gradient information of an image, deeplearning, or the like. The method of detecting the attributes of aperson is not limited to those described above, and an arbitrarydetection method may be applied.

The information amount reduction degree determiner 258 determines adegree of reduction of the amount of information (for example, ablurring intensity) at the time of reducing the amount of information ofa person image on the basis of attributes detected by the attributedetector 257 (in other words, attributes of a person shown in theimage). For example, the information amount reduction degree determiner258 determines a blurring intensity on the basis of a degree ofappearance of an attribute of a person. Here, the degree of appearanceof an attribute, for example, is an appearance frequency based on thenumber of times of appearance in the image captured by the camera 10.For example, in a case in which there is a one person wearing redclothing among five persons, an appearance frequency of red clothing isone and becomes 20%. In a case in which there are three persons wearingblue clothing among five persons, an appearance frequency of blueclothing is three and becomes 60%. The information amount reductiondegree determiner 258 determines a blurring intensity to be “strong” fora person image having an attribute of a low appearance frequency suchthat the degree of reduction of the amount of information becomeshigher. Here, the blurring intensity being “strong” is a level set inadvance as an amount of blurring for which it is difficult to identifyindividuals. On the other hand, the information amount reduction degreedeterminer 258 determines a blurring intensity to be “weak” for a personimage having an attribute of a high appearance frequency such that thedegree of reduction of the amount of information becomes lower. Here,the blurring intensity being “weak” is a level for which the amount ofblurring is smaller than that of “strong” and is a level set in advancefor the purpose of inhibiting reduction of collation accuracy.

Here, it may be set in advance that an appearance frequency of anattribute is high, and an appearance frequency of another attribute islow. For example, it may be set in advance that an appearance frequencyof an attribute is high, and an appearance frequency of anotherattribute is low on the basis of results of investigation acquired inadvance from images in the past. In addition, an appearance frequencyacquired through estimation based on a season, a place, presence/absenceof an event, details of an event, a fashion, and the like may be set inadvance. For example, for a job conference and the like, it can beestimated that there are many persons wearing suits, and accordingly, anappearance frequency of persons wearing suits may be set to be high inadvance.

FIG. 6 is a diagram illustrating a setting of association between anappearance frequency of an attribute of a person and a blurringintensity. This diagram illustrates an example of a setting ofassociation between a color of clothing as an attribute and a blurringintensity for each color of clothing. In the example illustrated in thedrawing, for a person U1 wearing green clothing and a person U2 wearingred clothing, an appearance frequency is low (an appearance number oftimes is one), and identification of individuals is easy, andaccordingly, the blurring intensity is set to be “strong.” For personsU3, U4, and U5 wearing blue clothing, an appearance frequency is high(an appearance number of times is three), and, originally, it isdifficult to identify individuals, and accordingly, the blurringintensity is set to be “weak.” In this embodiment, although an examplein which two types of blurring intensity including “strong” and “weak”are used is described, the number of types of blurring intensity is notlimited to two, and three or more types of blurring intensity may beused in accordance with an appearance frequency and the like of anattribute.

Referring back to FIG. 5 , the information amount reducer 259 generatesimage data (information-reduced data) acquired by reducing at least apart of the amount of information of an image in accordance with adegree of reduction of the amount of information (for example, ablurring intensity) determined by the information amount reductiondegree determiner 258. For example, the information amount reducer 259performs blurring processing for a person image acquired by the personimage acquisitor 254 in accordance with a blurring intensity determinedby the information amount reduction degree determiner 258, therebygenerating data of an image for which the blurring processing has beenperformed (image data after the blurring processing).

The feature quantity calculator 260 calculates a person feature quantity(information reduced feature quantity) representing a feature quantityof a person from image data (information reduced data) acquired byperforming blurring processing for a person image using the informationamount reducer 259. This person feature quantity (an information reducedfeature quantity) is that of information reduced data, luminancegradient information calculated from the information reduced data, anoutput of deep learning having information reduced data as its input,and the like. In addition, the feature quantity calculator 260 transmitsthe calculated person feature quantity (the information reduced featurequantity) to the server 30. For example, the feature quantity calculator260 transmits the calculated person feature quantity (the informationreduced feature quantity), capture time information of the image, andidentification information of the camera 10 that has captured the imagein association with each other.

The feature quantity storage 351 receives a person feature quantity (aninformation reduced feature quantity) transmitted from the featurequantity calculator 260 and stores the received person feature quantityin the storage 34. For example, the feature quantity storage 351 storesa person feature quantity (an information reduced feature quantity)received from each of a plurality of cameras 10, capture timeinformation of an image, and identification information of the camera 10that has captured the image in the storage 34 in association with eachother.

The feature quantity collator 352 collates a plurality of person featurequantities (information reduced feature quantities) received from theplurality of cameras 10, which are stored by the feature quantitystorage 351, with each other. As a method of collating person featurequantities with each other, a method using an inner product of twoperson feature quantities, a Euclid distance between two person featurequantities, an absolute value of a difference between two person featurequantities, or the like can be used. The feature quantity collator 352associates feature quantities determined as the same person on the basisof a result of the collation.

The display controller 353 causes the display 33 to display informationbased on a result of the collation acquired by the feature quantitycollator 352. For example, the display controller 353 displaysstatistical information such as features, behavior patterns, and thelike of persons.

Next, operations of Re-id processing of associating same personscaptured by a plurality of cameras 10 in the information processingsystem 1 will be described with reference to FIG. 7 . FIG. 7 is aflowchart illustrating one example of Re-id processing according to thisembodiment.

The terminal 20 acquires images captured by the camera 10 (Step S100).The terminal 20 detects person images of persons shown in the imagesfrom the images acquired from the camera 10 and acquires data of personimages acquired by cutting out areas of the person images (Step S110).Next, the terminal 20 executes blurring processing for the person images(Step S120).

FIG. 8 is a flowchart illustrating details of the blurring processing ofStep S120.

The terminal 20 detects attributes of persons on the basis of the personimages detected from the images acquired from the camera 10. Forexample, in the example illustrated in FIG. 6 , the terminal 20 detectsa color of clothing of a person from the person image (Step S1211).Next, the terminal 20 determines a blurring intensity on the basis ofthe detected attributes of the person. For example, in the exampleillustrated in FIG. 6 , the terminal 20 determines the blurringintensity for a person image in which a color of clothing is red orgreen to be “strong” and determines a blurring intensity for a personimage in which a color of clothing is blue to be “weak” (Step S1212).Then, the terminal 20 generates image data acquired by blurring theperson image with the determined blurring intensity (information reduceddata) (Step S1213).

Referring back to FIG. 7 , the terminal 20 calculates a person featurequantity from the image data after the blurring processing generated inStep S1213 (Step S130). Then, the terminal 20 transmits the calculatedperson feature quantity to the server 30 (Step S140).

When the person feature quantity transmitted from the terminal 20 isreceived (Step S200), the server 30 stores the received person featurequantity in the storage 34 (Step S210). The server 30 collates aplurality of person feature quantities, which are stored, received froma plurality of cameras 10 with each other and associates featurequantities determined as the same person on the basis of results of thecollation (Step S220).

Then, the server 30 outputs information based on the results of thecollation. For example, the server 30 causes the display 33 to displaystatistical information of features, behavior patterns, and the like ofpersons and the like (Step S230).

As described above, the information processing system 1 according tothis embodiment determines a blurring intensity (one example of a degreeof reduction of the amount of information) on the basis of attributes ofa person (one example of a subject) shown in an image and generatesinformation reduced data acquired by reducing at least a part of theamount of information of the image in accordance with the determinedblurring intensity. In this way, the information processing system 1 canappropriate reduce an amount of information of an image in accordancewith attributes of a subject shown in the image.

In this embodiment, although an example in which a blurring intensity isdetermined on the basis of one attribute (for example, a color ofclothing) has been described, a blurring intensity may be determined onthe basis of a plurality of attributes.

In addition, the information processing system 1 calculates aninformation reduction feature quantity representing a feature quantityof a person (one example of a subject) from the generated informationreduced data and collates calculated information reduced featurequantities with each other. In this way, the information processingsystem 1 can detect the same person from information reduced data whilereducing the amount of information of an image on the basis ofattributes of a person (one example of a subject) such that it isdifficult to identify individuals.

For example, the information processing system 1 determines a blurringintensity (one example of a degree of reduction of the amount ofinformation) on the basis of an appearance frequency (one example of adegree of appearance) of an attribute of a person (one example of asubject). In other words, the information processing system 1 determineswhether or not it is easy or difficult to identify a person inside animage on the basis of an appearance frequency of an attribute of aperson (one example of a subject) and changes the blurring intensity.The information processing system 1 sets the blurring intensity to bestrong for a person image of a person in which an appearance frequencyof the same attribute is low (a person who can be easily identified),whereby the privacy is taken into consideration. On the other hand, theinformation processing system 1 sets the blurring intensity to be weakfor a person image of a person for which an appearance frequency of thesame attribute is high (a person who cannot be easily identified),whereby a decrease in the collation accuracy is inhibited. Byappropriately setting the blurring intensity for each person image, adecrease in the accuracy of collation between persons whose features aresimilar is inhibited, and a decrease in the collation accuracy can beinhibited as being minimal while the privacy is taken intoconsideration.

In addition, a degree of appearance such as an appearance number oftimes may be used instead of the appearance frequency.

In addition, in the information processing system 1, the terminal 20transmits the calculated information reduced feature quantity to theserver 30 through a communication network. In this way, data transmittedto the server 30 through a communication network becomes a data fromwhich it is difficult to identify individuals, and accordingly, aconfiguration in consideration of privacy can be formed.

Second Embodiment

Next, a second embodiment will be described.

A basic configuration of an information processing system 1 according tothis embodiment is similar to the configuration illustrated in FIGS. 4and 5 , and thus description thereof will be omitted. In thisembodiment, a blurring intensity is determined for each part of aperson, which is different from the first embodiment. For example, thereare cases in which an appearance frequency is different for each partsuch as a hairstyle, clothing of an upper body half, clothing of a lowerbody half, or the like. In such cases, by increasing the amount ofinformation for a part of which an appearance frequency is high anddecreasing the amount of information for a part of which an appearancefrequency is low, collation of which accuracy is higher than that of acase in which a single blurring intensity is used can be performed whilethe privacy is taken into consideration.

For example, a person detector 253 detects a person image for each partof a person shown in an image from the image acquired by an imageacquisitor 251. Detection of a part can be performed using resultsacquired by a person detector (a face detector, an upper body halfdetector, or a person detector), instance segmentation, a semanticsegmentation, and the like. A person image acquisitor 254 acquires apartial image for each part of a person detected by the person detector253.

An attribute detector 257 detects attributes from a partial image foreach part of a person on the basis of a person image acquired by theperson image acquisitor 254. An information amount reduction degreedeterminer 258 determines a blurring intensity on the basis of anattribute for each partial image. An information amount reducer 259generates image data (an information reduced data) acquired by reducingthe amount of information for each partial image in accordance with ablurring intensity for each partial image in a personal image.

FIG. 9 is a diagram illustrating an example of a setting of a blurringintensity for each part of a person according to this embodiment. Inthis drawing, an example in which attributes are detected for each ofthree parts including hair, a color of clothing, a color of trousers asattributes, and a blurring intensity is determined in accordance with anappearance frequency of each of the attributes is illustrated. In theexample illustrated in the drawing, a person U11 having long hair and aperson U15 wearing a hat have low appearance frequencies (the number oftimes of appearance is one), and individual identification thereof canbe easily performed. Accordingly, a blurring intensity for the hair isset to be “strong.” On the other hand, persons U12, U13, and U14 havingshort hair have high appearance frequencies (the number of times ofappearance is three), and, originally, individual identification thereofcannot be easily performed. Accordingly, a blurring intensity for thehair is set to be “weak.” In addition, the blurring intensity for thishair may be regarded as a blurring intensity for a head (face) part. Inaddition, the person U11 wearing red clothing and the person U12 wearinggreen clothing have low appearance frequencies (the number of times ofappearance is one), and individual identification thereof can be easilyperformed. Accordingly, a blurring intensity for the clothing is set tobe “strong.” On the other hand, the persons U13, U14, and U15 wearingblue clothing have high appearance frequencies (the number of times ofappearance is three), and, originally, individual identification thereofcannot be easily performed. Accordingly, a blurring intensity for theclothing is set to be “weak.” In addition, the person U13 wearing greentrousers has low an appearance frequency (the number of times ofappearance is one), and individual identification thereof can be easilyperformed. Accordingly, a blurring intensity for the trousers is set tobe “strong.” On the other hand, the persons U11, U12, U14, and U15wearing blue trousers have high appearance frequencies (the number oftimes of appearance is four), and, originally, individual identificationthereof cannot be easily performed. Accordingly, blurring intensitiesfor the trousers are set to be “weak.” In this way, for each part, ablurring intensity for the part is determined in accordance with anappearance frequency of an attribute of each part.

FIG. 10 is a flowchart illustrating blurring processing according tothis embodiment. The blurring processing illustrated in the drawing is aprocess executed in the blurring processing of Step S120 illustrated inFIG. 7 .

The terminal 20 detects attributes of a person for each part on thebasis of a person image. For example, in the example illustrated in FIG.9 , the terminal 20 detects hair, a color of clothing, and a color oftrousers of a person from a person image (Step S1221).

Next, the terminal 20 determines a blurring intensity on the basis ofthe attributes of the person detected for each part. For example, in theexample illustrated in FIG. 9 , the terminal 20 determines a blurringintensity for a hair part (or a head (face) part) in the person image tobe “strong” in the case of a person image having long hair or a hat anddetermines a blurring intensity to be “weak” in the case of a personimage having short hair. In addition, the terminal 20 determines ablurring intensity for a clothing part in the person image to be“strong” in a case of the person image in which the color of clothing isred or green and determines a blurring intensity to be “weak” in a caseof the person image in which the color of clothing is blue. Furthermore,the terminal 20 determines a blurring intensity for a trousers part (alower body half part) in the person image to be “strong” in a case of aperson image in which the color of trousers is green and determines theblurring intensity to be “weak” in a case of a person image in which thecolor of trousers is blue (Step S1222).

Then, the terminal 20 generates image data (information reduced data)acquired by blurring each part of the person image with the determinedblurring intensity for each part (Step S1223).

As described above, the information processing system 1 according tothis embodiment determines a blurring intensity (one example of a degreeof reduction of the amount of information) on the basis of an attributefor each of one or more partial images of a person image and generatesinformation reduced data acquired by reducing the amount information foreach partial image in accordance with a blurring intensity for eachpartial image. In this way, the information processing system 1 sets theamount of information to be large for a part having a high appearancefrequency and sets the amount of information to be small for a parthaving a low appearance frequency. Accordingly, collation with higheraccuracy than that of a case in which a single blurring intensity isused can be performed while the privacy is taken into consideration.

Third Embodiment

Next a third embodiment will be described.

A basic configuration of an information processing system 1 according tothis embodiment is similar to the configuration illustrated in FIGS. 4and 5 , and thus description thereof will be omitted. In the first andsecond embodiments, a blurring intensity is determined in accordancewith an appearance frequency set in advance. However, there are cases inwhich an appearance frequency of an attribute changes in accordance withelapse of time. In such cases, by changing a blurring intensity inaccordance with an appearance frequency that changes in accordance withthe elapse of time, collation with higher accuracy than that of a casein which the blurring intensity is fixed can be performed. Thus, in thisembodiment, a configuration in which a blurring intensity is adaptivelychanged in accordance with an appearance frequency that changes inaccordance with the elapse of time will be described.

FIG. 11 is a diagram illustrating an example in which a blurringintensity is adaptively changed according to this embodiment. Forexample, at a time T, an appearance frequency of each of persons U1, U2,and U3 whose colors of clothing are green, red, and blue is one, and allthe blurring intensities are “strong.” At a time T+ΔT after that, it isassumed that appearance frequencies of only the persons U3, U4, and U5whose colors of clothing are blue are increased. In this case, in a casein which a blurring intensity for a person image in which the color ofclothing is blue of which the appearance frequency has been increased isset to be “strong,” there is a high likelihood that the collationaccuracy decreases. For this reason, when an increase in the appearancefrequency of a person whose color of clothing is blue is detected at thetime T+ΔT, the terminal 20 changes the blurring intensity for the personimage in which the color of clothing is blue to “weak.”

FIG. 12 is a flowchart illustrating blurring processing according tothis embodiment. The blurring processing illustrated in the drawing is aprocess executed in the blurring processing of Step S120 illustrated inFIG. 7 .

The terminal 20 detects attributes of a person on the basis of a personimage detected from an image acquired from the camera 10. For example,in the example illustrated in FIG. 6 , the terminal 20 detects a colorof clothing of a person from a person image (Step S1231). The terminal20 stores the detected attribute of the person (for example, a color ofclothing) in the storage 24 (Step S1232). The terminal 20 occasionallycalculates and updates an appearance frequency of each attribute inaccordance with elapse of time on the basis of attributes (for example,colors of clothing) of a plurality of persons stored in the storage 24(Step S1233). The terminal 20 determines a blurring intensity of adetected attribute (for example, a color of clothing) of a person on thebasis of the latest appearance frequency that has been updated (StepS1234). Then, the terminal 20 generates image data (information reduceddata) acquired by blurring the person image with the determined blurringintensity (Step S1235).

In addition, as described in the second embodiment, also in a case inwhich a blurring intensity is determined for each part of a person, theterminal 20 may adaptively change a blurring intensity for each part inaccordance with an appearance frequency of an attribute for each partthat changes in accordance with elapse of time.

As described above, the information processing system 1 according tothis embodiment adaptively changes a blurring intensity in accordancewith an appearance frequency that changes in accordance with elapse oftime, and accordingly, collation with higher accuracy than that of acase in which the blurring intensity is fixed can be performed.

In addition, by detecting an appearance frequency of an attribute fromthe start, a blurring intensity may be adaptively changed.Alternatively, a blurring intensity may be initially set to be “strong”or “weak,” and thereafter, the blurring intensity may be adaptivelychanged in accordance with the appearance frequency that changes inaccordance with elapse of time. In a case in which a blurring intensityis set to be “strong” as an initial setting, protection of privacy isprioritized. On the other hand, in a case in which a blurring intensityis set to be “weak” as an initial setting, collation accuracy isprioritized.

Fourth Embodiment

Next a fourth embodiment will be described.

A basic configuration of an information processing system 1 according tothis embodiment is similar to the configuration illustrated in FIGS. 4and 5 , and thus description thereof will be omitted. As described inthe third embodiment, in a case in which a blurring intensity isadaptively changed in accordance with an appearance frequency of anattribute that changes in accordance with elapse of time, correctcollation may not be able to be performed in collation between a featurequantity calculated from a person image blurred with a blurringintensity after the change and a feature quantity calculated from aperson image blurred with a blurring intensity before the change.Although collation between only feature quantities before change andcollation between only feature quantities after change can be performed,the blurring intensity is different in collation between before andafter a change point. Accordingly, feature quantities are changed alsofor the same person, and correct collation may not be able to beperformed.

For example, as illustrated in FIG. 11 , in a case in which a blurringintensity for a person image in which a color of clothing is blue is setto “strong” at a time T but is changed to “weak” at a time T+ΔT inaccordance with an increase in the appearance frequency, collationbetween a person image in which a color of clothing is blue at the timeT and a person image in which a color of clothing is blue at the timeT+ΔT cannot be correctly performed. Thus, in this embodiment, asillustrated in FIG. 13 , in a case in which image data (informationreduced data) acquired by blurring a person image of persons U3, U4, andU5 whose colors of clothing are blue with a blurring intensity “weak”determined in accordance with an appearance frequency is generated, theterminal 20 also generates image data (information reduced data)acquired by blurring the person image with a blurring intensity “strong”in addition thereto. FIG. 13 is a diagram illustrating a blurringintensity according to this embodiment. In other words, the informationamount reducer 259 generates image data (information reduced data)acquired by blurring with a blurring intensity determined in accordancewith an appearance frequency of an attribute from a person image andadditionally generates image data (information reduced data) acquired byblurring with a blurring intensity higher than the blurring intensity.

FIG. 14 is a flowchart illustrating blurring processing according tothis embodiment. The blurring processing illustrated in the drawing is aprocess executed in the blurring process of Step S120 illustrated inFIG. 7 . Processes of Steps S1241 to S1244 illustrated in FIG. 14 aresimilar to the processes of Steps S1231 to S1234 illustrated in FIG. 12. Here, operations of processes of Step S1245 and subsequent steps willbe described.

The terminal 20 determines whether or not there has been a change in ablurring intensity determined in Step S1244 from a blurring intensitythat was previously determined (Step S1245). In a case in which it isdetermined that there has been no change in Step S1245 (No), theterminal 20 generates image data (information reduced data) acquired byblurring a person image with the blurring intensity determined in StepS1244. For example, in a case in which there has been no change in theblurring intensity with being maintained to be “strong,” the terminal 20generates image data (information reduced data) acquired by blurring aperson image with the blurring intensity “strong” (Step S1246).

On the other hand, in a case in which it is determined that there hasbeen a change in Step S1245 (Yes), the terminal 20 generates image data(information reduced data) acquired by blurring the person image withthe blurring intensity determined in Step S1244 and image data(information reduced data) acquired by blurring the person image withthe blurring intensity before the change. For example, in a case inwhich the blurring intensity has been changed from “strong” to “weak,”the terminal 20 generates image data (information reduced data) acquiredby blurring the person image with a blurring intensity “weak” and imagedata (information reduced data) acquired by blurring the person imagewith the blurring intensity “strong” before the change (Step S1247).

As described above, the information processing system 1 according tothis embodiment generates image data (information reduced data) acquiredby blurring with a blurring intensity (for example, a blurring intensity“weak”) determined in accordance with an appearance frequency of anattribute a person image and additionally generates image data(information reduced data) acquired by blurring with a blurringintensity (for example, a blurring intensity “strong”) higher than theblurring intensity. In this way, in a case in which the blurringintensity is changed, the information processing system 1 can performcollation before and after a change.

In addition, in a case in which the blurring intensity has been changedfrom “weak” to “strong,” the terminal 20 may generate image data(information reduced data) acquired by blurring with a blurringintensity “strong” and image data (information reduced data) acquired byblurring with the blurring intensity “weak” (Step S1247) In this way, ina case in which the blurring intensity is changed in accordance with achange in the appearance frequency, the terminal 20 may generate imagedata (information reduced data) acquired by blurring a person image witha blurring intensity before the change in addition to generation ofimage data (information reduced data) acquired by blurring the personimage with a blurring intensity after the change.

In addition, in a case in which the blurring intensity is determined tobe “weak” regardless of presence/absence of a change in the blurringintensity, the terminal 20 may generate image data (information reduceddata) acquired by blurring a person image with a blurring intensity“strong” before change in addition to the generation of image data(information reduced data) acquired by blurring the person image with ablurring intensity “weak.” To the contrary, in a case in which theblurring intensity is determined to be “strong” regardless ofpresence/absence of a change in the blurring intensity, the terminal 20may generate image data (information reduced data) acquired by blurringa person image with a blurring intensity “weak” before change inaddition to the generation of image data (information reduced data)acquired by blurring the person image with a blurring intensity“strong.” In addition, the terminal 20 may generate both image data(information reduced data) acquired by blurring a person image with ablurring intensity “strong” and image data (information reduced data)acquired by blurring the person image with a blurring intensity “weak”regardless of presence/absence of a change in the blurring intensity.

Fifth Embodiment

Next, a fifth embodiment will be described.

FIG. 15 is a diagram illustrating a blurring intensity according to thisembodiment. As described in the third embodiment, in a case in which theblurring intensity is adaptively changed in accordance with anappearance frequency of an attribute that changes in accordance withelapse of time, there is a time difference between a time at which theappearance frequency has been changed and a time at which the blurringintensity is changed in accordance with the change. In the exampleillustrated in the drawing, at a time T, an appearance frequency of eachof persons U1, U2, and U3 whose colors of clothing are green, red, andblue is one, and all the blurring intensities are “strong.” At a timeT+ΔT after that, although appearance frequencies of only persons U3, U4,and U5 whose colors of clothing are blue are increased, the blurringintensity of any one thereof is maintained to be “strong” in accordancewith a time difference until the blurring intensity is changed. Inaccordance with this time difference, when the blurring intensity for aperson image in which the color of clothing is blue is originally to bechanged to “weak” in accordance with the increase in the appearancefrequency as illustrated in FIG. 11 , there is a time in which imagedata (information reduced data) acquired by blurring with an intendedblurring intensity cannot be generated.

Thus, in this embodiment, the terminal 20 has a function for storing anacquired image for a predetermined time and generates image data(information reduced data) acquired by blurring with an intendedblurring intensity from a timing at which there has been a change in theappearance frequency by retroactively applying the blurring intensitythat has been changed with a delay in accordance with the timedifference to stored images. The basic configuration of an informationprocessing system 1 according to this embodiment is similar to theconfiguration illustrated in FIG. 4 , and a part of the functionalconfiguration is different from the configuration illustrated in FIG. 4.

FIG. 16 is a block diagram illustrating one example of the functionalconfigurations of a terminal controller 25A and a server controller 35according to this embodiment. The terminal controller 25A is afunctional configuration according to this embodiment corresponding tothe terminal controller 25 illustrated in FIGS. 4 and 5 . The terminalcontroller 25A further includes an image storage 252A, which isdifferent from the functional configuration of the terminal controller25 illustrated in FIG. 5 . The image storage 252A stores images acquiredby an image acquisitor 251 in a storage 24 for a predetermined time. Inaddition, since there is a highest risk of leakage of an image in acommunication part from the terminal 20 to a server 30, by storingimages only in the terminal 20, the risk of leakage of images can beinhibited, and the privacy can be considered.

In a case in which the blurring intensity is changed in accordance witha change in the appearance frequency of an attribute, an informationamount reducer 259 regenerates image data (information reduced data)acquired by blurring a person image acquired from stored images with ablurring intensity after change. Then, a feature quantity calculator 260recalculates a person feature quantity from image data (informationreduced data) that has been regenerated and transmits the calculatedperson feature quantity to the server 30. In this way, the blurringintensity after change can be applied to an image from a time at whichthe appearance frequency has been changed by going back a timedifference between a time at which the appearance frequency has beenchanged and a time at which the blurring intensity is changed inaccordance with the change.

FIG. 17 is a flowchart illustrating blurring processing according tothis embodiment. The blurring processing illustrated in the drawing is aprocess executed in the blurring process of Step S120 illustrated inFIG. 7 . Processes of Steps S1251 to S1254 illustrated in FIG. 17 aresimilar to the processes of Steps S1231 to S1234 illustrated in FIG. 12. Here, operations of processes of Step S1255 and subsequent steps willbe described.

The terminal 20 determines whether or not there has been a change in ablurring intensity determined in Step S1254 from a blurring intensitythat was previously determined (Step S1255). In a case in which it isdetermined that there has been no change in the blurring intensity inStep S1255 (No), the terminal 20 generates image data (informationreduced data) acquired by blurring a person image with the blurringintensity determined in Step S1254 For example, in a case in which therehas been no change in the blurring intensity determined in Step S1254with being maintained to be “strong,” the terminal 20 generates imagedata (information reduced data) acquired by blurring a person image withthe blurring intensity “strong” (Step S1259).

On the other hand, in a case in which it is determined that there hasbeen a change in the blurring intensity in Step S1255 (for example, theblurring intensity determined in Step S1254 has been changed from“strong” to “weak”) (Yes), the terminal 20 acquires an image (forexample, an image acquired from a time at which the appearance frequencyhas been changed) that has been stored (Step S1256). Then, the terminal20 detects a person image from the acquired image and acquires data of aperson image acquired by cutting out an area of the person image (StepS1257). In addition, the terminal 20 detects an attribute (for example,a color of clothing) of a person from the acquired person image. Then,the terminal 20 generates image data (information reduced data) acquiredby blurring a person image acquired from the storage image with theblurring intensity “weak” (Step S1259).

As described above, the information processing system 1 according tothis embodiment has a function of storing an image acquired from thecamera 10 and generates information reduced data acquired by reducing atleast a part of the amount of information of the stored image inaccordance with the changed blurring intensity in a case in which theblurring intensity (one example of a degree of reduction of the amountof information) is changed on the basis of an attribute of a person (oneexample of a subject) shown in the image. In this way, the informationprocessing system 1 can apply the blurring intensity after change to animage from a time at which the appearance frequency has been changed bygoing back a time difference between a time at which the appearancefrequency has been changed and a time at which the blurring intensity ischanged in accordance with the change.

Sixth Embodiment

Next, a sixth embodiment will be described.

A basic configuration of an information processing system 1 according tothis embodiment is similar to the configuration illustrated in FIG. 4 ,and thus description thereof will be omitted. In a case in which theblurring intensity is adaptively changed in accordance with anappearance frequency of an attribute changing in accordance with theelapse of time, it is determined that an attribute of a person captureda plurality of number of times has an appearance frequency higher thanthat of an attribute of a person who is captured only once. For example,as illustrated in FIG. 18 , it is determined that persons U1 and U2 whohave passed through the inside of a capture range (a capture fieldangle) of the camera 10 only once have a low appearance frequency, andthe blurring intensity is set to “strong.” On the other hand, there arecases in which a person U3 who has passed through the inside of thecapture range (the capture field angle) of the camera 10 any number oftimes, who is an actually one person and is to be determined to have alow appearance frequency, is determined to have a high appearancefrequency, and the blurring intensity is set to “weak.” In other words,there may be a deviation between an actual appearance frequency of anattribute and an appearance frequency of the attribute calculated fromthe number of times of capturing, and the blurring intensity may not beable to be correctly determined.

Thus, in this embodiment, the terminal 20 calculates a feature quantityfrom an image before blurring processing, and performs collation withhigh accuracy for identifying an individual, whereby the same person whohas been captured a plurality of number of times is identified. FIG. 19is a system diagram illustrating an overview of the informationprocessing system 1 according to this embodiment. In the exampleillustrated in the drawing, collation for identifying the same person isperformed before blurring processing using the terminal 20, which isdifferent from the example illustrated in FIG. 1 . Since the terminal 20identifies the same person and calculates an appearance frequencywithout doubly counting an appearance frequency of an attribute of thesame person, a deviation between the actual appearance frequency of anattribute and an appearance frequency of the attribute calculated fromthe number of times of capturing can be inhibited. In addition, thefeature quantity from which an individual can be identified is placedonly inside the terminal 20 but is not transmitted to the server 30, andaccordingly, the privacy can be considered.

FIG. 20 is a block diagram illustrating one example of the functionalconfigurations of a terminal controller 25B and a server controller 35according to this embodiment. The terminal controller 25B is afunctional configuration according to this embodiment that correspondsto the terminal controller 25 illustrated in FIGS. 4 and 5 . Theterminal controller 25B further includes a pre feature quantitycalculator 255B (a second feature quantity calculator) and a pre featurequantity storing and collator 256B (a second collator), which isdifferent from the functional configuration of the terminal controller25 illustrated in FIG. 5 .

The pre feature quantity calculator 255B calculates a person featurequantity (a feature quantity before reduction of the amount ofinformation) representing a feature quantity of a person from a personimage acquired by the person image acquisitor 254 (in other words, aperson image before blurring processing). This person feature quantity(a feature quantity before reduction of the amount of information) isthat of a person image acquired by the person image acquisitor 254 (inother words, a person image before blurring processing), luminancegradient information calculated from the person image, an output of deeplearning having the person image as its input, and the like. Forexample, the pre feature quantity calculator 255B outputs the calculatedperson feature quantity (a feature quantity before reduction of theamount of information), the capture time information of the image, andidentification information of the person image to the pre featurequantity storing and collator 256B in association with each other.

The pre feature quantity storing and collator 256B stores the personfeature quantity (the feature quantity before reduction of the amount ofinformation) output from the pre feature quantity calculator 255B,capture time information of the image, and the identificationinformation of the person image in the storage 24 in association witheach other. In addition, the pre feature quantity storing and collator256B collates stored person feature quantities (feature quantitiesbefore reduction of amount of information) with each other. As a methodof collating person feature quantities with each other, a method usingan inner product of two person feature quantities, a Euclid distancebetween two person feature quantities, an absolute value of a differencebetween two person feature quantities, or the like can be used. The prefeature quantity storing and collator 256B associates feature quantitiesdetermined as the same person on the basis of results of collation withthe calculated identification information of the person image, therebyidentifying person images of the same person. Then, the pre featurequantity storing and collator 256B outputs the same person informationincluding the same persons and identification information of theidentified person images to the information amount reduction degreedeterminer 258.

The information amount reduction degree determiner 258 calculates anappearance frequency of an attribute on the basis of the same personinformation output from the pre feature quantity storing and collator256B and the attribute detected by the attribute detector 257 (in otherwords, an attribute of a person shown in the image). For example, theinformation amount reduction degree determiner 258 counts an appearancenumber of times of attributes of the same person among attributesdetected by the attribute detector 257 as one although the attributeshave appeared a plurality of number of times (captured a plurality ofnumber of times). In this way, the information amount reduction degreedeterminer 258 calculates an appearance frequency such that anappearance frequency of attributes of the same person is not doublycounted and determines a blurring intensity on the basis of thecalculated appearance frequency.

FIG. 21 is a flowchart illustrating blurring processing according tothis embodiment. The blurring processing illustrated in the drawing is aprocess executed in the blurring processing of Step S120 illustrated inFIG. 7 .

The terminal 20 calculates a person feature quantity (an featurequantity before reduction of the amount of information) from a personimage (in other words, a person image before blurring processing)acquired by the person image acquisitor 254 (Step S1261) and stores thecalculated person feature quantity (a feature quantity before reductionof the amount of information) in the storage 24 (Step S1262). Theterminal 20 collates stored person feature quantities (featurequantities before reduction of the amount of information) with eachother (Step S1263) and identifies person images of the same person onthe basis of results of the collation (Step S1264).

In addition, the terminal 20 detects an attribute (for example, a colorof clothing) of a person from a person image detected from an imageacquired from the camera 10 (Step S1265) and stores the detectedattribute of the person in the storage 24 (Step S1266). Then, theterminal 20 calculates an appearance frequency of an attribute on thebasis of person images identified as the same person and detectedattributes (in other words, attributes of a person shown in the image)such that an appearance frequency of the attributes of the same personis not doubly counted (Step S1267).

Then, the terminal 20 determines a blurring intensity of the detectedattribute of the person (for example, a color of clothing) on the basisof the calculated appearance frequency (Step S1268) and generates imagedata (information reduced data) acquired by blurring the person imagewith the determined blurring intensity (Step S1269).

As described above, the information processing system 1 according tothis embodiment calculates a person feature quantity (a feature quantitybefore reduction of the amount of information) of a person (one exampleof a subject) from an image before reduction of the amount ofinformation and collates the person feature quantities (featurequantities before reduction of the amount of information). Then, theinformation processing system 1 determines a blurring intensity (oneexample of an information reduction degree) on the basis of a result ofcollation between the attribute of the person (one example of thesubject) shown in the image and a person feature quantity beforereduction of the amount of information (a feature quantity beforereduction of the amount of information).

In accordance with this, in a case in which the same person reciprocatesthe inside of the capture range (the image field angle) of the camera 10any number of times and is captured a plurality of number of times, theinformation processing system 1 can identify the same person and cancount the appearance frequency without being doubly counted.Accordingly, a deviation between an actual appearance frequency of theattribute and an appearance frequency of the attribute calculated fromthe number of times of capturing can be inhibited. Accordingly, theamount of information of the image can be appropriately reduced.

In addition, the pre feature quantity storing and collator 256B mayoutput the same person information to the attribute detector 257 insteadof or in addition to the information amount reduction degree determiner258. In such a case, the attribute detector 257 transmits attributes ofperson images identified as the same person in association withinformation indicating the same person, whereby the information amountreduction degree determiner 258 may be configured not to doubly countthe appearance frequency of the attributes of the same person.

Modified Example

In the embodiment described above, although an example in which theinformation amount reduction degree determiner 258 selects anddetermines one among blurring intensities (“strong,” “weak,” and thelike) set in advance has been described, the blurring intensity may bedetermined through calculation. For example, the information amountreduction degree determiner 258 may determine a blurring intensitythrough calculation using a calculation equation for calculating thevalue of the blurring intensity and the like on the basis of theappearance frequency.

In the embodiment described above, although an example in which the sameperson is detected as an example of a case in which the subject is aperson has been described, the subject may be an object other than aperson. For example, the subject may be any one of various objects suchas a vehicle and an animal instead of the person. For example, thesubject is a moving body that can move and is an object (target object)focused inside a captured image. The detection of various objects suchas a vehicle and the like can be performed using results acquired by avehicle detector, a text detector, a general object detector, instancesegmentation, semantic segmentation, and the like. These can be realizedusing template matching, statistics of luminance gradient information ofan image, deep learning, or the like. For example, in a case in whichthe subject is a vehicle, attributes are a vehicle type and a color ofthe vehicle. The vehicle type can be estimated using vehiclerecognition. The color of the vehicle can be estimated using colorinformation (chromaticity and the like) of the image. Recognition of avehicle can be realized using template matching, statistics of luminancegradient information of an image, deep learning, and the like. Inaddition, also in a case in which the subject is a vehicle, the amountof information may be reduced by determining an information reductionquantity on the basis of attributes of each part such as a color of thevehicle, a vehicle type, a number plate, and the like. In accordancewith this, also in a case in which the subject is an object other than aperson, similarly, collation with higher accuracy than that of a case inwhich a single information reduction degree is used can be performedwhile the privacy is taken into consideration.

In addition, an image acquired by the image acquisitor 251 is notlimited to a color image but may be a grey image. Even in the greyimage, a type of clothing (a suite, trousers, half pants, or a skirt, orthe like), a hairstyle (long hair, short hair, or the like),presence/absence of a hat, presence/absence of glasses, and the like canbe determined. In addition, the color may be determined from the densityof the grey image. Furthermore, the image acquisitor 251 may acquire adistance image from a distance sensor.

In addition, the terminal 20 and the server 30 described above have acomputer system inside. By recording a program for realizing thefunction of each component included in the terminal 20 and the server 30described above in a computer-readable recording medium and causing acomputer system to read and execute the program recorded in thisrecording medium, the process of each component included in the terminal20 and the server 30 described above may be performed. Here, “causing acomputer system to read and execute the program recorded in therecording medium” includes installing the program in the computersystem. The “computer system” described here includes an operatingsystem (OS) and hardware such as peripherals. In addition, a “computersystem” may include a plurality of computer devices connected through anetwork including the Internet, a WAN, and a LAN and a communicationline such as a dedicated line. Furthermore, the “computer-readablerecording medium” represents a portable medium such as a flexible disk,a magneto-optical disk, a ROM, or a CD-ROM or a storage device such as ahard disk built in the computer system. In this way, the recordingmedium storing a program may be a non-transitory recording medium suchas a CD-ROM.

In addition, the recording medium includes a recording medium installedinside or outside that is accessible from a distribution server fordistributing the program. Furthermore, a configuration in which theprogram is divided into a plurality of parts, and the parts aredownloaded at different timings and then are combined in eachconfiguration included in the terminal 20 and the server 30 may beemployed, and distribution servers distributing the divided programs maybe different from each other. In addition, the “computer-readablerecording medium” includes a medium storing the program for apredetermined time such as an internal volatile memory (RAM) of acomputer system serving as a server or a client in a case in which theprogram is transmitted through a network. Furthermore, the programdescribed above may be a program used for realizing a part of thefunction described above. In addition, the program may be a program tobe combined with a program that has already been recorded in thecomputer system for realizing the function described above, a so-calleda differential file (differential program).

Furthermore, a part or the whole of each function included in theterminal 20 and the server 30 according to the embodiment describedabove may be realized by an integrated circuit of a large scaleintegration (LSI) or the like. Each function may be individuallyconfigured as a processor, or a part or the whole of the functions maybe integrated and configured as a processor. In addition, a techniqueused for configuring the integrated circuit is not limited to the LSI,and each function may be realized by a dedicated circuit or ageneral-purpose processor. Furthermore, in a case in which a technologyof configuring an integrated circuit replacing the LSI emerges inaccordance with the progress of semiconductor technologies, anintegrated circuit using such a technology may be used.

In addition, in the embodiment described above, although an example inwhich information processing system 1 is a server-client type systemincluding the terminal 20 and the server 30 has been described as anexample, the information processing system 1 may be configured as oneintegrated information processing device. In such a case, a componentcollating feature quantity may be included or may not be included. Inother words, a configuration until the amount of information is reducedwith a blurring intensity according to an appearance frequency (oneexample of an information reduction degree) for an image captured by thecamera 10 may be employed, and thereafter, a configuration in whichfeature quantities are calculated, and collation of the same person isperformed may be employed. In addition, the camera 10 is not limited toa plurality of cameras but may be one camera. More specifically, theinformation processing system may be applied to a monitoring device usedfor simply monitoring and recording passengers, visitors, and the likeusing one or a plurality of cameras installed in a facility such as astore or a building, a street such as a shopping street, a station yard,a parking lot, or the like.

According to at least one embodiment described above, by including theimage acquisitor (251) that acquires an image acquired by imaging anactual space, the information amount reduction degree determiner (258)that determines a degree of reduction of an amount of information on thebasis of an attribute of a subject shown in the image, the informationamount reducer (259) that generates information reduced data acquired byreducing at least a part of the amount of information of the image inaccordance with the degree of reduction of the amount of informationdetermined by the information amount reduction degree determiner, theamount of information of the image can be appropriately reduced.

While several embodiments of the present invention have been described,such embodiments are presented as examples but are not intended to limitthe scope of the present invention. These embodiments may be performedin other various forms, and various omissions, substitutions, andchanges may be performed in a range not departing from the concept ofthe present invention therein. These embodiments and the modificationsthereof, similar to a case where these are included in the scope or theconcept of the invention, are included in inventions described in theclaims and equivalent ranges thereof.

What is claimed is:
 1. An information processing device comprising: an image acquisitor that acquires an image acquired by imaging an actual space, the image including a plurality of subjects each of which includes a respective attribute; an information amount reduction degree determiner that determines, for each attribute, a respective degree of appearance of subjects having its attribute shown in the image, the respective degree of appearance of each attribute corresponding to a respective number of subjects having each attribute appearing in the image; the information amount reduction degree determiner adjusts, for each attribute, a respective degree of reduction of an amount of information of a respective subject based on its respective degree of appearance of the attribute, so that the degree of reduction of the amount of information is smaller as the respective degree of appearance is greater and so that the degree of reduction of the amount of information is larger as the respective degree of appearance is smaller; and an information amount reducer that generates information reduced data acquired by reducing at least a part of the amount of information of the image in accordance with the degree of reduction of the amount of information determined by the information amount reduction degree determiner.
 2. The information processing device according to claim 1, further comprising: a first feature quantity calculator that calculates an information reduced feature quantity representing a feature quantity of the subject from the information reduced data; and a first collator that collates the information reduced feature quantities.
 3. The information processing device according to claim 2, further comprising: a transmitter that transmits the information reduced feature quantity to the server through a communication network.
 4. The information processing device according to claim 1, wherein the information amount reduction degree determiner determines the degree of reduction of the amount of information on the basis of an attribute of each of one or more partial images of the image, and wherein the information amount reducer generates the information reduced data acquired by reducing the amount of information of the partial image in accordance with the degree of reduction of the amount of information of each partial image.
 5. The information processing device according to claim 1, wherein the information amount reducer generates the information reduced data according to the degree of reduction of the amount of information from the image and further generates the information reduced data according to a degree of reduction of the amount of information higher than this degree of reduction of the amount of information.
 6. The information processing device according to claim 1, wherein the image acquisitor has a function of storing the image, and wherein, in a case in which the information amount reduction degree determiner changes the degree of reduction of the amount of information on the basis of the attribute of the subject shown in the image, the information amount reducer generates the information reduced data acquired by reducing at least a part of the amount of information of the stored image in accordance with the degree of reduction of the amount of information changed by the information amount reduction degree determiner.
 7. The information processing device according to claim 1, further comprising: a second feature quantity calculator that calculates a feature quantity before reduction of the amount of information representing a feature quantity of the subject from the image before reduction of the amount of information using the information amount reducer; and a second collator that collates the feature quantities before reduction of the amount of information, wherein the information amount reduction degree determiner determines the degree of reduction of the amount of information on the basis of a result of collation between the attribute of the subject shown in the image and the feature quantity before reduction of the amount of information.
 8. The information processing device according to claim 1, wherein the information reduction degree is a blurring intensity at the time of reducing the amount of information by blurring the image, and wherein the information reduction data is image data acquired by performing blurring processing of at least a part of the image with this blurring intensity.
 9. The information processing device according to claim 2, further comprising: a terminal including the image acquisitor, the information amount reduction degree determiner, the information amount reducer, and the first feature quantity calculator; and a server including the first collator, wherein the terminal transmits the information reduced feature quantity calculated from the information reduced data to the server, and wherein the server receives the information reduced feature quantity from the terminal.
 10. The information processing device according to claim 9, wherein the terminal further includes: a second feature quantity calculator that calculates a feature quantity before reduction of the amount of information representing the feature quantity of the subject from the image; and a second collator that collates the feature quantities before reduction of the amount of information.
 11. An information processing system comprising: a terminal; and a server, wherein the terminal includes: an image acquisitor that acquires an image acquired by imaging an actual space, the image including a plurality of subjects each of which includes a respective attribute; an information amount reduction degree determiner that determines, for each attribute, a respective degree of appearance of subjects having its attribute shown in the image, the respective degree of appearance of each attribute corresponding to a respective number of subjects having each attribute appearing in the image; the information amount reduction degree determiner adjusts, for each attribute, a respective degree of reduction of an amount of information of a respective subject based on its respective degree of appearance of the attribute, so that the degree of reduction of the amount of information is smaller as the respective degree of appearance is greater and so that the degree of reduction of the amount of information is larger as the respective degree of appearance is smaller; an information amount reducer that generates information reduced data acquired by reducing at least a part of the amount of information of the image in accordance with the degree of reduction of the amount of information determined by the information amount reduction degree determiner; a feature quantity calculator that calculates an information reduced feature quantity representing a feature quantity of the subject from the information reduced data; and a transmitter that transmits the information reduced feature quantity to the server through a communication network, wherein the server includes: a receiver that receives the information reduced feature quantity transmitted from the server; and a collator that collates the information reduced feature quantities received by the receiver.
 12. An information processing method using an information processing device, the information processing method comprising: acquiring an image acquired by imaging an actual space, the image including a plurality of subjects each of which includes a respective attribute; determining, for each attribute, a respective degree of appearance of subjects having its attribute shown in the image, the respective degree of appearance of each attribute corresponding to a respective number of subjects having each attribute appearing in the image; adjusting, for each attribute, a respective degree of reduction of an amount of information of a respective subject based on its respective degree of appearance of the attribute, so that the degree of reduction of the amount of information is smaller as the respective degree of appearance is greater and so that the degree of reduction of the amount of information is larger as the respective degree of appearance is smaller; and generating information reduced data acquired by reducing at least a part of the amount of information of the image in accordance with the determined degree of reduction of the amount of information.
 13. A non-transitory computer readable storage medium that stores computer program for causing a computer to execute: acquiring an image acquired by imaging an actual space by using an image acquisitor, the image including a plurality of subjects each of which includes a respective attribute; determining, for each attribute, a respective degree of appearance of subjects having its attribute shown in the image, the respective degree of appearance of each attribute corresponding to a respective number of subjects having each attribute appearing in the image; adjusting, for each attribute, a respective degree of reduction of an amount of information of a respective subject based on its respective degree of appearance of the attribute, so that the degree of reduction of the amount of information is smaller as the respective degree of appearance is greater and so that the degree of reduction of the amount of information is larger as the respective degree of appearance is smaller; and generating information reduced data acquired by reducing at least a part of the amount of information of the image in accordance with the determined degree of reduction of the amount of information. 